On the similarity of symbol frequency distributions with heavy tails
نویسندگان
چکیده
Quantifying the similarity between symbolic sequences is a traditional problem in information theory which requires comparing the frequencies of symbols in different sequences. In numerous modern applications, ranging from DNA over music to texts, the distribution of symbol frequencies is characterized by heavy-tailed distributions (e.g., Zipf’s law). The large number of low-frequency symbols in these distributions posesmajor difficulties to the estimation of the similarity between sequences; e.g., theyhinder an accurate finite-size estimation of entropies. Here, we show analytically how the systematic (bias) and statistical (fluctuations) errors in these estimations depend on the sample sizeN and on the exponent γ of the heavy-tailed distribution. Our results are valid for the Shannon entropy (α 1⁄4 1), its corresponding similarity measures (e.g., the Jensen-Shanon divergence), and also for measures based on the generalized entropy of order α. For small α’s, including α 1⁄4 1, the errors decay slower than the 1=N decay observed in short-tailed distributions. For α larger than a critical value α 1⁄4 1þ 1=γ ≤ 2, the 1=N decay is recovered. We show the practical significance of our results by quantifying the evolution of the English language over the last two centuries using a completeα spectrumofmeasures.We find that frequent words changemore slowly than less frequent words and that α 1⁄4 2 provides the most robust measure to quantify language change.
منابع مشابه
Similarity of symbol frequency distributions with heavy tails
Quantifying the similarity between symbolic sequences is a traditional problem in Information Theory which requires comparing the frequencies of symbols in different sequences. In numerous modern applications, ranging from DNA over music to texts, the distribution of symbol frequencies is characterized by heavy-tailed distributions (e.g., Zipf’s law). The large number of low-frequency symbols i...
متن کاملHeavy-tailed Probability Distributions in the World Wide Web
The explosion of the World Wide Web as a medium for information dissemination has made it important to understand its characteristics, in particular the distribution of its le sizes. This paper presents evidence that a number of le size distributions in the Web exhibit heavy tails, including les requested by users, les transmitted through the network, transmission durations of les, and les stor...
متن کاملHeavy - Tailed Probability
The explosion of the World Wide Web as a medium for information dissemination has made it important to understand its characteristics, in particular the distribution of its le sizes. This paper presents evidence that a number of le size distributions in the Web exhibit heavy tails, including les requested by users, les transmitted through the network, transmission durations of les, and les stor...
متن کاملEvaluation and Application of the Gaussian-Log Gaussian Spatial Model for Robust Bayesian Prediction of Tehran Air Pollution Data
Air pollution is one of the major problems of Tehran metropolis. Regarding the fact that Tehran is surrounded by Alborz Mountains from three sides, the pollution due to the cars traffic and other polluting means causes the pollutants to be trapped in the city and have no exit without appropriate wind guff. Carbon monoxide (CO) is one of the most important sources of pollution in Tehran air. The...
متن کاملModeling financial data with stable distributions
Stable distributions are a class of probability distributions that allow heavy tails and skewness. In addition to theoretical reasons for using stable laws, they are a rich family that can accurately model different kinds of financial data. We review the basic facts, describe programs that make it practical to use stable distributions, and give examples of these distributions in finance. A non-...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1510.00277 شماره
صفحات -
تاریخ انتشار 2015